Skip to content

test: split slow tests, cache CI deps, add pre-push gate#83

Merged
bartzbeielstein merged 1 commit into
mainfrom
perf/accelerate-tests
Jun 1, 2026
Merged

test: split slow tests, cache CI deps, add pre-push gate#83
bartzbeielstein merged 1 commit into
mainfrom
perf/accelerate-tests

Conversation

@bartzbeielstein
Copy link
Copy Markdown
Contributor

Context

The pytest suite has grown to 1,727 tests across 151 files and takes ~113 s locally with -n auto; on GitHub it's worse because no dependency caching exists, so every run reinstalls all deps including the CPU-only torch wheel. There was also no way to skip heavy tests (no markers) and no enforcement of running tests before pushing.

This PR accelerates the suite locally and in CI without removing any test coverage, and adds a local pre-push gate.

What changed

Slow/fast split (coverage preserved)

  • New tests/conftest.py central registry tags the 109 heaviest end-to-end tests @pytest.mark.slow (derived from pytest --durations). No tests deleted — the full suite still runs everything.
  • pyproject.toml: registered the slow marker + addopts=["--durations=15"].
Run Tests Wall-clock (local)
Full suite 1,727 ~113 s
Fast gate (-m "not slow") 1,618 ~48 s

CI acceleration (ci.yml, release.yml)

  • Dependency caching via setup-uv enable-cache (existing pinned action SHA kept) — torch & friends stop re-downloading every run.
  • Installs switched uv pip install --systemuv sync (no more double-install with uv run).
  • Fast required PR gate (-m "not slow") + new test-full job running the whole suite nightly (cron) and on workflow_dispatch. release.yml now runs the fast gate as a pre-publish check instead of the full suite.

Force tests before push

  • New .pre-commit-config.yaml: pre-push hook runs uv run pytest -m "not slow" -n auto and blocks the push on failure; commit-stage ruff + black --check. Added pre-commit to dev deps.
  • Install per-clone (documented in README): uv run pre-commit install --hook-type pre-push --hook-type pre-commit.

Notes for reviewers

  • The slow set lives in one reviewable place (tests/conftest.py) and is matched by node id (file / class / test granularity); refresh it from --durations as the suite evolves. New slow tests can also just use @pytest.mark.slow.
  • uv.lock includes a pre-existing lockfile refresh that was already in the working tree (Dependabot transitive bumps) plus the new pre-commit dependency tree.
  • Real CI timing improvements will be visible on this PR's first run (warm cache benefits subsequent runs).

Verification

  • Fast suite: 1,618 passed in ~48 s; slow set: 109 tests deselected.
  • ruff check + black --check clean on new files; all workflow/precommit YAML parses.
  • Pre-commit hooks install and validate; commit-stage hooks pass (verified on the commit in this PR).

🤖 Generated with Claude Code

Accelerate the test suite locally and on GitHub Actions without removing
coverage.

- Tag the 109 heaviest end-to-end tests `@pytest.mark.slow` via a central
  registry in tests/conftest.py (derived from `pytest --durations`). The
  fast path (`-m "not slow"`) runs 1618 tests in ~48s vs ~113s for the full
  suite; nothing is deleted and the full suite still runs everything.
- CI (ci.yml): enable setup-uv dependency caching (torch & friends stop
  re-downloading every run), switch installs to `uv sync`, and split into a
  fast required PR gate plus a nightly/on-demand `test-full` job that runs
  the whole suite. release.yml runs the fast gate as a pre-publish check.
- Add a pre-commit pre-push hook (.pre-commit-config.yaml) that runs the
  fast suite and blocks the push on failure; document install in README.
- Register the `slow` marker and `--durations=15`; add `pre-commit` dev dep.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@bartzbeielstein bartzbeielstein merged commit d643d38 into main Jun 1, 2026
3 of 4 checks passed
@bartzbeielstein bartzbeielstein deleted the perf/accelerate-tests branch June 4, 2026 10:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant